Evaluation of Random Forest Model

Row

Evaluation of Accuracy

Row

Cross-Validation Accuracy vs Number of Random Predictors

Confusion Matrix

$table
          Reference
Prediction    A    B    C    D    E
         A 1674    0    0    0    0
         B    7 1127    5    0    0
         C    0    5 1019    2    0
         D    0    0    7  954    3
         E    0    0    0    2 1080

$overall
      Accuracy          Kappa  AccuracyLower  AccuracyUpper   AccuracyNull 
     0.9947324      0.9933361      0.9925313      0.9964182      0.2856415 
AccuracyPValue  McnemarPValue 
     0.0000000            NaN 

Evaluation of Boosting Model

Accuracy vs Boosting Iterations

Confusion Matrix

$table
          Reference
Prediction    A    B    C    D    E
         A 1656   10    3    5    0
         B   38 1060   39    2    0
         C    0   35  979    9    3
         D    1    4   32  918    9
         E    7   11    9   15 1040

$overall
      Accuracy          Kappa  AccuracyLower  AccuracyUpper   AccuracyNull 
  9.605777e-01   9.501083e-01   9.552874e-01   9.654049e-01   2.892099e-01 
AccuracyPValue  McnemarPValue 
  0.000000e+00   7.638490e-09 

$byClass
         Sensitivity Specificity Pos Pred Value Neg Pred Value Precision
Class: A   0.9729730   0.9956969      0.9892473      0.9890762 0.9892473
Class: B   0.9464286   0.9834208      0.9306409      0.9873578 0.9306409
Class: C   0.9218456   0.9902550      0.9541910      0.9829183 0.9541910
Class: D   0.9673340   0.9906807      0.9522822      0.9937005 0.9522822
Class: E   0.9885932   0.9913097      0.9611830      0.9975016 0.9611830
            Recall        F1 Prevalence Detection Rate Detection Prevalence
Class: A 0.9729730 0.9810427  0.2892099      0.2813934            0.2844520
Class: B 0.9464286 0.9384683  0.1903144      0.1801189            0.1935429
Class: C 0.9218456 0.9377395  0.1804588      0.1663551            0.1743415
Class: D 0.9673340 0.9597491  0.1612574      0.1559898            0.1638063
Class: E 0.9885932 0.9746954  0.1787596      0.1767205            0.1838573
         Balanced Accuracy
Class: A         0.9843349
Class: B         0.9649247
Class: C         0.9560503
Class: D         0.9790074
Class: E         0.9899515

Evaluation of Support Vector Machine Model

Row

Diagnostics for Linear Support Vector Machine

Support Vector Machine object of class "ksvm" 

SV type: C-svc  (classification) 
 parameter : cost C = 1 

Linear (vanilla) kernel function. 

Number of Support Vectors : 7225 

Objective Function Value : -1450.051 -1275.306 -1046.239 -628.1268 -1326.171 -881.7941 -1774.954 -1207.525 -1037.249 -1209.01 
Training error : 0.210526 

Diagnostics for Radial Support Vector Machine

Support Vector Machine object of class "ksvm" 

SV type: C-svc  (classification) 
 parameter : cost C = 1 

Gaussian Radial Basis kernel function. 
 Hyperparameter : sigma =  0.0138342908497241 

Number of Support Vectors : 7042 

Objective Function Value : -1128.491 -836.3012 -734.9416 -438.009 -1048.849 -589.373 -761.5491 -1012.77 -718.7094 -628.9886 
Training error : 0.068501 

Row

Confusion Matrix Linear Support Vector Machine

$table
          Reference
Prediction    A    B    C    D    E
         A 1550   27   34   55    8
         B  143  824   68   22   82
         C   92   94  804   19   17
         D   68   30  112  707   47
         E   73  131   70   52  756

$overall
      Accuracy          Kappa  AccuracyLower  AccuracyUpper   AccuracyNull 
  7.886151e-01   7.310874e-01   7.779569e-01   7.989865e-01   3.272727e-01 
AccuracyPValue  McnemarPValue 
  0.000000e+00   3.543329e-53 

Confusion Matrix Radial Support Vector Machine

$table
          Reference
Prediction    A    B    C    D    E
         A 1660    5    7    2    0
         B   87  997   52    1    2
         C    4   41  956   24    1
         D    6    3  105  849    1
         E    6   13   57   24  982

$overall
      Accuracy          Kappa  AccuracyLower  AccuracyUpper   AccuracyNull 
  9.250637e-01   9.050519e-01   9.180390e-01   9.316635e-01   2.995752e-01 
AccuracyPValue  McnemarPValue 
  0.000000e+00   2.353217e-41